Skip to content

Greylilac09/fix mem tracking#3

Closed
GreyLilac09 wants to merge 188 commits into
masterfrom
greylilac09/fix-mem-tracking
Closed

Greylilac09/fix mem tracking#3
GreyLilac09 wants to merge 188 commits into
masterfrom
greylilac09/fix-mem-tracking

Conversation

@GreyLilac09
Copy link
Copy Markdown
Owner

Summary

Vector configuration

How did you test this PR?

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

Notes

  • Please read our Vector contributor resources.
  • Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
  • Some CI checks run only after we manually approve them.
    • We recommend adding a pre-push hook, please see this template.
    • Alternatively, we recommend running the following locally before pushing to the remote branch:
      • cargo fmt --all
      • cargo clippy --workspace --all-targets -- -D warnings
      • cargo nextest run --workspace (alternatively, you can run cargo test --all)
  • After a review is requested, please avoid force pushes to help us review incrementally.
    • Feel free to push as many commits as you want. They will be squashed into one before merging.
    • For example, you can run git merge origin master and git push.
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run cargo vdev build licenses to regenerate the license inventory and commit the changes (if any). More details here.

pront and others added 30 commits August 28, 2025 14:56
* fix(ci): features check

* fix for scripts/check-one-feature api
* chore(deps): update VRL

* chore(dev): cargo vdev build licenses
Update SMP CLI version and build type for regression suite

Signed-off-by: Caleb Metz <caleb.metz@datadoghq.com>
* Fix tests failing with cargo test

* Use tokio Mutex instead

* Implement global lock as suggested in PR review

* Fix clippy

* Drop guard after binding

* Rename wait_for_tcp->wait_for_tcp_and_release

* Lock when using log_schema in tests

* Lock in tracing-limit's tests due to tracing::subcriber::with_default

* Revert "Lock when using log_schema in tests"

This reverts commit eed7a3878a5c0c74b3e759c8c470d42b353964d7.

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>
…ordotdev#23662)

* feat(config): log changed global fields when reload is rejected

* fix markdown

* cargo fmt

---------

Co-authored-by: Thomas <thomas.schneider@datadoghq.com>
* feat(new source): okta

* add changelog fragment

* docs

* Update website/cue/reference/components/sources/base/okta.cue

Co-authored-by: Heston Hoffman <hestonhoffman@gmail.com>

* Update website/cue/reference/components/sources/base/okta.cue

Co-authored-by: Heston Hoffman <hestonhoffman@gmail.com>

* Update website/cue/reference/components/sources/okta.cue

Co-authored-by: Heston Hoffman <hestonhoffman@gmail.com>

* refactor url mutex

* rename

* fix nits

* cargo fmt

* rename

* rephrase

* Fix spelling in changelog

* Add okta source to semantic.yml

* Apply docs suggestions to the correct file

* Fix rust 2024

* Move file from base into generated

* Creatively fix spelling

* Even more creative spelling fixes

* Fix invalid `base` reference

* Fix most clippy issues

* Remove useless assignment

* Allow too_many_arguments

* tests

* website/scripts/cue.sh fmt

* chore(vdev): apply vdev rust check fixes

* cargo fmt

---------

Co-authored-by: Heston Hoffman <hestonhoffman@gmail.com>
Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>
Co-authored-by: Thomas <thomasqueirozb@gmail.com>
Co-authored-by: Thomas <thomas.schneider@datadoghq.com>
* fix(dev): fix sources-okta component features check

* Improve check-one-feature script
…go group (vectordotdev#23674)

* chore(deps): Bump tracing-subscriber in the cargo group

Bumps the cargo group with 1 update: [tracing-subscriber](https://github.com/tokio-rs/tracing).


Updates `tracing-subscriber` from 0.3.19 to 0.3.20
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](tokio-rs/tracing@tracing-subscriber-0.3.19...tracing-subscriber-0.3.20)

---
updated-dependencies:
- dependency-name: tracing-subscriber
  dependency-version: 0.3.20
  dependency-type: direct:production
  dependency-group: cargo
...

Signed-off-by: dependabot[bot] <support@github.com>

* cargo vdev build license

* Make tracing-subscriber a workspace dependency

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Thomas <thomas.schneider@datadoghq.com>
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.20.0 to 3.21.0.
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](Stebalien/tempfile@v3.20.0...v3.21.0)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-version: 3.21.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [indexmap](https://github.com/indexmap-rs/indexmap) from 2.10.0 to 2.11.0.
- [Changelog](https://github.com/indexmap-rs/indexmap/blob/main/RELEASES.md)
- [Commits](indexmap-rs/indexmap@2.10.0...2.11.0)

---
updated-dependencies:
- dependency-name: indexmap
  dependency-version: 2.11.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [actions/cache](https://github.com/actions/cache) from 4.2.3 to 4.2.4.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](actions/cache@5a3ec84...0400d5f)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-version: 4.2.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.17.0 to 1.18.0.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](uuid-rs/uuid@v1.17.0...v1.18.0)

---
updated-dependencies:
- dependency-name: uuid
  dependency-version: 1.18.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(vdev): prepare for crates.io

* tweaks
Bumps [notify](https://github.com/notify-rs/notify) from 8.1.0 to 8.2.0.
- [Release notes](https://github.com/notify-rs/notify/releases)
- [Changelog](https://github.com/notify-rs/notify/blob/main/CHANGELOG.md)
- [Commits](notify-rs/notify@notify-8.1.0...notify-8.2.0)

---
updated-dependencies:
- dependency-name: notify
  dependency-version: 8.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ordotdev#23524)

* wip

* WIP2

* wip3

* ran cargo fmt

* refactoring otel source

* wip

* ran cargo fmt

* commit the otel proto desc

* refactoring

* fix re-build mess

* unrelated web-playground build fix

* avoid duplication in tests

* dbg tests

* dbg tests

* ran cargo fmt

* chore(dev): cargo vdev build licenses

* wip - cleanup

* ran cargo fmt

* chore(deps): cargo update -p vrl

* chore(dev): cargo vdev build licenses

* e2e - wip

* cleanup

* ran cargo fmt

* set default

* wip

* vdev cruft

* ran cargo fmt

* cleanup

* fix new config

* use JSON names - wip

* ran cargo fmt

* chore(dev): cargo vdev build licenses

* update VRL

* ran cargo fmt

* chore(dev): cargo vdev build licenses

* handle metrics too

* ran cargo fmt

* vrl update fixes

* ran cargo fmt

* wip

* chore(dev): cargo vdev build licenses

* fix typo and add first version of the changelog

* ran cargo fmt

* fix typo and add first version of the changelog

* add author

* Apply suggestions from code review

Co-authored-by: Ursula Chen <58821586+urseberry@users.noreply.github.com>

* tweaks to changelog

* gen comp docs

* fix changelog typo

---------

Co-authored-by: Ursula Chen <58821586+urseberry@users.noreply.github.com>
)

Bumps [syslog_loose](https://github.com/FungusHumungus/syslog-loose) from 0.22.0 to 0.23.0.
- [Changelog](https://github.com/StephenWakely/syslog-loose/blob/master/CHANGELOG.md)
- [Commits](StephenWakely/syslog-loose@v0.22.0...v0.23.0)

---
updated-dependencies:
- dependency-name: syslog_loose
  dependency-version: 0.23.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* feat(vdev): introduce release workflow

* fix tag format

* fix fmt

* rmv extra newline

* remove defaults and update pkg-url to use the new tag pattern
…dev#23699)

* chore(ci): Bump github/codeql-action from 3.29.7 to 3.30.0

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.29.7 to 3.30.0.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@51f7732...2d92b76)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 3.30.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update .github/workflows/scorecard.yml

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Thomas <thomas.schneider@datadoghq.com>
…#23710)

Bumps [docker/login-action](https://github.com/docker/login-action) from 3.4.0 to 3.5.0.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](docker/login-action@74a5d14...184bdaa)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: 3.5.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ev#23691)

Bumps [netlink-packet-core](https://github.com/rust-netlink/netlink-packet-core) from 0.7.0 to 0.8.0.
- [Release notes](https://github.com/rust-netlink/netlink-packet-core/releases)
- [Changelog](https://github.com/rust-netlink/netlink-packet-core/blob/main/CHANGELOG)
- [Commits](rust-netlink/netlink-packet-core@v0.7.0...v0.8.0)

---
updated-dependencies:
- dependency-name: netlink-packet-core
  dependency-version: 0.8.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>
* chore(ci): simplify bootstrap-macos.sh

* revert cargo.lock changes
)

Bumps [actions/checkout](https://github.com/actions/checkout) from 4.2.2 to 5.0.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@11bd719...08c6903)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 5.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [sysinfo](https://github.com/GuillaumeGomez/sysinfo) from 0.36.1 to 0.37.0.
- [Changelog](https://github.com/GuillaumeGomez/sysinfo/blob/master/CHANGELOG.md)
- [Commits](GuillaumeGomez/sysinfo@v0.36.1...v0.37.0)

---
updated-dependencies:
- dependency-name: sysinfo
  dependency-version: 0.37.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…v#23684)

Bumps [security-framework](https://github.com/kornelski/rust-security-framework) from 3.2.0 to 3.3.0.
- [Release notes](https://github.com/kornelski/rust-security-framework/releases)
- [Commits](kornelski/rust-security-framework@v3.2.0...v3.3.0)

---
updated-dependencies:
- dependency-name: security-framework
  dependency-version: 3.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
pront and others added 19 commits October 10, 2025 13:41
…dotdev#23984)

* fix(ci): latest-vector_default.yaml case was silently failing

* review: add rustdocs

* fix service name and format

* ran cargo fmt
…ing (vectordotdev#23986)

In vectordotdev#13176, the `datadog_agent` source had functionality introduced to
automatically split the metric series name into namespace and name components on
the first period in that name.  This change was done to address vectordotdev#12953 where a
default namespace was configured being applied to a metrics sink, resulting in
metrics from this source having an undesired prefix added.

Since all of the metric sinks will reproduce the metric series name based on
both the namespace and name in the same way, this is a lossless transform.
However, if we want to filter on the full name (ex in a remap VRL script etc),
having the namespace split off becomes a problem.

To address this problem, this change introduces a `split_metric_namespace`
option to the `datadog_agent` source. It defaults to `true`, which is the
current behavior, so the change is entirely backward compatible with existing
configs.

The bulk of the size of the change comes from the addition of new tests and
refactoring existing tests to reduce duplicated code.
…ectordotdev#23977)

* fix(ci): scripts/run-integration-test.sh must fail early (not skip)

* Update scripts/upload-test-results.sh

Co-authored-by: Thomas <thomas.schneider@datadoghq.com>

* discovered failing test: allow empty attributes, recent regression

* compare protobuf instances directly instead of doing brittle custom comparison

support batching

ran cargo fmt

* ran cargo fmt

* atempt to address fmt discrepancy across OSes

* add retries to prevent race condition, needed a wrapper image since telemetrygen is distroless

* revert changes to files (already submitted a separate PR)

---------

Co-authored-by: Thomas <thomas.schneider@datadoghq.com>
* feat(ci): parrelize misc tests

* replace ci-run-misc

* improve names
…otdev#23823)

* Implement proper indexer ack query compression in `splunk_hec` sinks

* Changelog fragment

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>
* chore(ci): remove redundant setup steps

* Fix typo

* apt conf settings in a single file

---------

Co-authored-by: Thomas <thomas.schneider@datadoghq.com>
…3996)

* fix(aws_s3 source): add retry delay in sqs::Ingestor

This commit adds logic to wait 500ms between subsequent calls of
run_once if it returns an error. This is required to prevent spamming
the SQS API if it fails to receive messages, e.g. due to invalid IAM
permissions or an SQS outage.

* add exp backoff

* add backoff tests

* add changelog

* ran cargo fmt

* fix api usage

* ran cargo fmt

* spellchecker allow

* Apply suggestion from @thomasqueirozb

Co-authored-by: Thomas <thomas.schneider@datadoghq.com>

* Apply suggestion from @thomasqueirozb

Co-authored-by: Thomas <thomas.schneider@datadoghq.com>

* Apply suggestion from @thomasqueirozb

Co-authored-by: Thomas <thomas.schneider@datadoghq.com>

* Fix event

---------

Co-authored-by: Adam Medzinski <medzin@users.noreply.github.com>
Co-authored-by: Thomas <thomas.schneider@datadoghq.com>
…dotdev#24002)

* feat(codecs): add 'use_json_names' options to protobuf codecs

* changelog

* update docs

* docs updates

* fix docs

* add job description field and enhance test

* clippy fixes

* docs feedback
* feat(codecs): introduce OTLP decoder and use it internally for the OTEL source

* make fmt

* add changelog

* generate docs

* clippy fix

* make fmt

* code golfing and tests

* fix schema definition

* make fmt

* remove unusused import

* improve docs

* improve test: validate trace ID and spand ID

* fmt linux
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.